1. Field of the Invention
The present invention relates to computer security, and deals more particularly with improving intrusion detection using sensitivity/suspicion levels for filtering intrusion events and/or using generic attack signatures as intrusion detection policy.
2. Description of the Related Art
In the computer security field, “intrusion” is a broad term encompassing many undesirable activities. The objective of an intrusion may be to acquire information that a person is not authorized to have (referred to as “information theft”). It may be to cause business harm by rendering a network, system, or application unusable (referred to as “denial of service”). Or, it may be to gain unauthorized use of a system as a stepping stone for further intrusions elsewhere. Most intrusions follow a pattern of information gathering, attempted access, and then destructive attacks.
Some attacks can be detected and neutralized by the target system, although not typically in real time. Other attacks cannot be effectively neutralized by the target system. Most of the attacks also make use of “spoofed” packets which are not easily traceable to their true origin. Many attacks now make use of unwitting accomplices—that is, machines or networks that are used without authorization to hide the identity of the attacker. For these reasons, detecting information gathering, access attempts, and attack accomplice behaviors is a vital part of intrusion detection. (The term “attack” is sometimes used in the art as a distinct type of intrusion, as contrasted to a “scan” or “flood”. As used herein, the term “attack” refers generally to all of these types of intrusions, except where specific ones of these intrusions are being described for illustrative purposes.)
As illustrated in FIG. 1, attacks can be initiated from outside the internal network (see elements 130, 135) or from inside the internal network (see elements 110, 115). Particularly vulnerable is an open system such as a public web server or any machine (see element 100) that is placed in service to serve those outside the internal network 115. A firewall 120 can provide some level of protection against attacks from outside. However, it cannot prevent attacks once the firewall has “approved” entry of a host into the internal network (as shown at 125), nor can it provide protection in the case where the attack is initiated from inside the network (as in the case of the attacker at 110). In addition, end-to-end encryption limits the types of attacks that can be detected by an intermediate device such as a firewall because the intermediate device is unable to evaluate the packets in cleartext for evidence of an attack.
An Intrusion Detection System (hereinafter, “IDS”) can provide detection of many types of attacks. Common IDS types (illustrated in FIG. 2) currently deployed include sniffers and scanners. Sniffers operate in promiscuous mode, examining traffic that passes through on the local network. Sniffers are placed at strategic points in the network, such as in front of a firewall (see elements 210, 220); behind a firewall (see elements 220, 230); in the network (see element 240); or in front of a host (see elements 250, 260). Sniffers use “pattern matching” to try to match a packet against a known attack which is expressed as an “attack signature”. Sniffers work best against single packet attacks: upon detecting that an inspected packet matches a particular attack signature, that packet can be dealt with appropriately (as further described below).
Commercial services such as IBM's Emergency Response Services (see element 200) provide third-party logging and analysis of security alerts detected by a customer's IDS components. In the example of FIG. 2, the sniffer 210 placed before the customer's firewall 220 sends its alerts to this type of third-party alert processing service.
Scanners (see element 270) look at log files for signs of attacks, which may be detected by inspecting a collection of packets. Scanners therefore do not inspect data packets as they are passing through the network. Instead, they operate in a post-processing mode.
The network sniffers and scanners which are currently deployed have a number of limitations. Five of these limitations are:
1. Network sniffers (such as elements 210, 230, 240 in FIG. 2) cannot evaluate data that is encrypted unless the sniffer and the encryption endpoint are located at the same node. Because sniffers are not typically co-located with the packet destination, providing an encryption endpoint at the sniffer is contradictory with the goal of end-to-end encryption (that is, encryption where the endpoints are the client and application server).
2. Processing signature files against every packet is very processor intensive. Many sniffers cannot process all packets, dropping packets that cannot be processed. Some installations deploy multiple sniffers, dividing the signature files into parts which are distributed to the individual sniffers. However, the ability to take real-time action in response to detecting a match against an attack signature still decreases as the backlog of traffic to analyze becomes larger.
3. Network sniffers typically cannot deflect the attacking packet or directly perform other preventative measures when an attack is detected. Thus, a damaging attack might occur even when a network sniffer detects the attack. (Sniffers and scanners typically take no direct action to stop the attacking packets, but rather notify an IDS management system. An IDS management system cannot provide a real-time reaction to prevent damage since the offending packet may reach the target system before the IDS system can respond.)
4. Scanners (such as element 270 in FIG. 2) work by evaluating logs for statistical anomalies and known attacks. Scanners have the ability to detect attacks missed by sniffers because they can react to a pattern of behavior. Because of their reliance on patterns over time, they are not real time. Furthermore, the post-processing analysis may in some cases result in “false positives”—that is, a conclusion that an attack has occurred when in reality the traffic was not attack packets.
5. Current attack signatures are written to cover single specific attacks. This results in IDSs not being able to detect new attacks until a new signature is developed and installed. Since the attack signatures are very specific, as new attacks are discovered, the signature lists can become large. As the signature lists increase, processing time at the IDS also increases, resulting in more missed attacks due to the IDS's failure to keep up with the incoming packet stream. Additionally, the IDS must be reloaded more often to pick up new signatures.
When an attack on a target system occurs, the damage may be extremely costly to repair. Accordingly, what is needed are improved techniques for performing intrusion detection which can increase the security of a target server or system.